Low-Resource Hate Speech Detection in English-Swahili Code-Switched Text Using Fine-Tuning of Pre-trained Language Models

Abstract

The use of social media in East Africa has grown rapidly, and with it, the spread of hate speech has become a serious concern. This problem is even more complex in online spaces where people often switch between English and Swahili within the same sentence or conversation. Such code-switching makes it difficult for existing systems to accurately detect harmful content, especially because there is limited labeled data and much of the language used is informal and context-dependent. This study explores a low-resource approach to detecting hate speech in English and Swahili code-switched text by fine-tuning pre-trained language models. In this work, transformer-based models such as BERT and AfriBERTa are adapted to better understand mixed-language communication. The models are trained on a carefully prepared dataset made up of real social media posts that reflect how people actually write and speak online. These posts are manually labeled to capture both direct and subtle forms of hate speech, including expressions that are influenced by local culture and everyday slang. The findings show that fine-tuned models perform better than traditional machine learning approaches, especially in terms of accuracy and overall detection quality. They are also more effective at handling informal language, abbreviations, and mixed grammar structures. Beyond performance, the study also looks at fairness and bias, emphasizing the need for systems that are sensitive to cultural and linguistic diversity. Overall, this work shows that fine-tuning modern language models can offer a practical and scalable solution for hate speech detection in multilingual environments.

Introduction

This study investigates hate speech detection in English–Swahili code-switched social media text in East Africa, where multilingual communication is common. As social media use has grown, harmful online content targeting people based on ethnicity, religion, gender, and political affiliation has also increased. Detecting such hate speech is difficult because users frequently mix English and Swahili, use slang (such as Sheng), abbreviations, emojis, and informal grammar. Existing NLP systems are largely trained on English data, while annotated datasets for African languages remain scarce.

The study proposes using fine-tuned transformer-based language models to improve hate speech detection in this low-resource multilingual setting. It compares transformer models with traditional machine learning methods while also evaluating fairness and bias in multilingual and culturally diverse contexts.

The literature review highlights that:

• Traditional machine learning methods (e.g., SVM and logistic regression) rely on features such as bag-of-words and n-grams but struggle with contextual meaning, sarcasm, slang, and code-switching.
• Code-switching presents major NLP challenges due to mixed grammar, inconsistent spelling, slang, and multilingual vocabulary.
• Transformer models such as BERT and AfriBERTa better capture contextual meaning through self-attention and bidirectional learning, making them more effective for multilingual hate speech detection.

For the methodology, the researchers collected 8,158 publicly available English–Swahili social media posts containing natural code-switching. The dataset was manually annotated into three categories: HATE, OFFENSIVE, and NEUTRAL. Annotation quality was validated using Cohen's Kappa (κ = 0.95), indicating almost perfect agreement between annotators.

During preprocessing, the data was cleaned by removing URLs and special characters, normalising multilingual text, tokenising using model-specific tokenisers, and handling emojis, slang, and abbreviations. Language identification and contextual analysis techniques were also applied to detect language switching, improving the dataset's quality and enabling more accurate hate speech classification in multilingual social media content.

Conclusion

This study demonstrated that fine-tuned transformer models are effective for detecting hate speech in English–Swahili code-switched text in low-resource settings. The results showed that BERT and AfriBERTa outperformed traditional machine learning models, with AfriBERTa achieving the best performance due to its strong representation of African languages. These findings highlight the potential of transformer-based models for multilingual hate speech detection in African social media. Future research should focus on developing larger and more diverse datasets, incorporating additional African languages, investigating real-time deployment for content moderation, and applying bias mitigation and explainable AI techniques to improve fairness, transparency, and model reliability.

References

[1] A. M. Badel, T. Zhong, X. Xu, W. Tai, and F. Zhou, “Hate speech detection in Somali-English code-switched texts,” in Proceedings of the CCF International Conference on Natural Language Processing and Chinese Computing, pp. 364–376, Springer Nature Singapore, 2025. [2] S. Das, A. Dutta, K. Roy, A. Mondal, and A. Mukhopadhyay, “A survey on automatic online hate speech detection in low-resource languages,” arXiv preprint arXiv:2411.19017, 2024. [Online]. Available: https://arxiv.org/abs/2411.19017 [3] T. Davidson, D. Warmsley, M. Macy, and I. Weber, “Automated hate speech detection and the problem of offensive language,” in Proceedings of the International AAAI Conference on Web and Social Media, vol. 11, no. 1, pp. 512–515, 2017. [4] J. Devlin, M. W. Chang, K. Lee, and K. Toutanova, “BERT: Pre-training of deep bidirectional transformers for language understanding,” in Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 4171–4186, Association for Computational Linguistics, 2019. [5] V. Marivate et al., “PuoBERTa: Training and evaluation of a curated language model for Setswana,” in Proceedings of the Southern African Conference for Artificial Intelligence Research, pp. 253–266, Springer, 2023. [6] S. H. Muhammad, I. Abdulmumin, A. A. Ayele, D. I. Adelani, I. S. Ahmad, S. M. Aliyu, et al., “AfriHate: A multilingual collection of hate speech and abusive language datasets for African languages,” in Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pp. 1854–1871, Association for Computational Linguistics, 2025. [7] N. S. Mullah, W. M. N. W. Zainon, F. Ridzuan, and E. O. Abiodun, “Machine learning intervention on cyber-hate in code-switch texts: A systematic review with open challenges and solutions,” PeerJ Computer Science, vol. 12, p. e3537, 2026, doi: 10.7717/peerj-cs.3537. [8] K. Ogueji, Y. Zhu, and J. Lin, “Small data? No problem! Exploring the viability of pretrained multilingual language models for low-resource languages,” in Proceedings of the 1st Workshop on Multilingual Representation Learning, pp. 116–126, Association for Computational Linguistics, 2021. [9] E. Ombui, L. Muchemi, and P. Wagacha, “Hate speech detection in code-switched text messages,” in Proceedings of the 2019 3rd International Symposium on Multidisciplinary Studies and Innovative Technologies (ISMSIT), pp. 1–6, IEEE, 2019. [10] N. O. Onyango, L. Wanzare, and J. Obuhuma, “Swahili and code-switched English-Swahili political hate speech detection textual dataset,” 2025. [11] S. Rananga, A. Modupe, A. Isong, and V. Marivate, “Misinformation detection: A review for high and low resource languages,” 2024. [12] H. Rathnayake, J. Sumanapala, R. Rukshani, and S. Ranathunga, “Adapter-based fine-tuning of pre-trained multilingual language models for code-mixed and code-switched text classification,” Knowledge and Information Systems, vol. 64, no. 7, pp. 1937–1966, 2022, doi: 10.1007/s10115-022-01698-1. [13] Solorio, Thamar, et al. \"Overview for the first shared task on language identification in code-switched data.\" Proceedings of the first workshop on computational approaches to code switching. 2014.

Copyright

Copyright © 2026 Kipkebut Andrew, Jepkemei Betty. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.